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Introduction 


Goals  of  the  literature  review: 

The  focus  of  the  literature  review  is  consumer  assessment  of  providers  at  the  medical 
group  level. 

The  Internet  provides  access  to  a  range  of  searchable  databases.  Without  a  search 
strategy,  however,  untold  hours  may  be  spent  scouring  different,  sometimes  overlapping 
bibliographies.  For  this  reason,  we  devised  a  systematic  search  strategy  to  ensure  a  quality  and 
comprehensive  search.  Time  limits  and  other  constraints  on  the  search  helped  to  combat 
diminishing  returns. 

Thorough  documentation  is  the  best  guarantee  that  references  are  not  lost  or  unnecessarily 
duplicated.  To  maintain  quality  control  in  our  literature  review,  we  documented  not  only  the 
reference  of  interest,  but  also  any  information  about  the  search  path  that  led  to  the  reference.  This 
helped  to  retrospectively  analyze  the  comprehensiveness  of  our  search  and  allows  for  the 
reproducibility  of  the  search  results. 

For  each  iteration  of  the  literature  search,  we  recorded  the  following  information:  a) 
database  searched,  b)  key  words  and  search  option  combinations,  c)  specific  organization  or 
WWW  site,  d)  years  used  for  the  search,  e)  number  of  returned  references  per  search,  and  f) 
number  of  selected  references  per  search. 

We  used  five  online  databases  for  the  literature  search:  MEDLINE,  HEALTHSTAR, 
ERIC,  JSTOR,  and  UNCOVER  (See  Appendix  A  for  a  description  of  each  database  used). 

Our  search  began  with  the  combinations  of  keywords  and  databases  outlined  in  the  search 
strategy  document  (see  Appendix  A).  We  conducted  45  different  searches.  Once  these  queries 
were  exhausted,  we  screened  the  results  based  upon  the  following  criteria:  articles  were  selected 
that  contained  partial,  full  or  referenced  instruments  that  measured  satisfaction  with  medical 
groups,  health  plans,  or  individual  physician  care.  This  included  review  papers  and  papers  that 
analyzed  primary  and  secondary  data  sources.  Studies  that  focused  on  hospital  satisfaction  were 
ignored. 

A  team  member  screened  them  initially  (and  liberally)  and  then  two  other  team  members 
then  screened  these  selections  and  a  set  of  articles  was  agreed  upon  and  ordered.  A  database  was 
created  to  track  the  articles  upon  various  categories  (level  of  assessment,  data  source,  and 
instrument  source).  These  articles  were  summarized  individually  and  compiled  by  project  staff. 

To  assess  the  comprehensiveness  of  the  literature  review,  experts  in  the  field  of  consumer 
assessment  of  health  plans  and  consumer  assessment  of  physicians  selected  a  set  of  important 
background  papers.  Those  involved  in  the  literature  search  were  blind  to  this  list,  and  the  search 
was  tested  to  see  if  the  papers  were  included.  After  the  first  iteration  of  our  search,  we  identified 


some  important  omissions  from  our  literature  search.  Articles  by  key  contributors  to  the  field  of 
patient  satisfaction  were  missing,  so  we  decided  to  begin  a  second  iteration  of  searches. 

The  project  team  devised  a  set  of  keywords  based  on  the  MESH  headings  from  the  set  of 
missing  articles.  These  searches  (n  =  15)  retrieved  both  the  missing  articles  and  others  on  related 
topics  (See  Appendix  A).  Members  of  the  team  conducted  these  14  additional  searches,  mirroring 
the  screening  and  selection  process  from  the  first  iteration,  and  compiling  the  summaries  from 
selected  articles. 

In  this  report  we  summarize  data  on  the  articles  that  include  information  on  consumer 
assessment  of  the  physician  or  group  of  physicians  at  the  physician  group  level.  We  present 
information  on  the  various  data  collection  methods  and  instruments  used  in  these  studies,  major 
dimensions  or  domains  assessed  (See  Table  1),  and  information  on  the  reliability  and  validity  of 
measures  (Table  2).  Finally,  where  available,  we  document  case  mix  variables  and  their  relations 
to  patient  evaluations  of  care.  (For  information  on  the  databases  reviewed  and  keyword  search 
strategy,  see  Appendix  A.  For  detailed  information  on  each  of  the  articles  reviewed,  please  see 
Appendix  B). 

We  reviewed  the  literature  with  the  following  goals  in  mind:  a)  to  identify  the  major 
dimensions  studied  related  to  patient  evaluations  of  the  provider  or  physician  group;  b)  to 
determine  which  instruments  are  being  used  most  frequently  to  assess  patient  experiences  with  the 
provider  or  physician  group;  c)  to  evaluate  the  state  of  the  art  in  measuring  patient  experiences 
with  care  and  provide  data  on  reliability  and  validity  of  instruments;  and  d)  to  review  data 
collection  strategies  used  in  gathering  patient  evaluations. 

Several  project  staff  reviewed  all  of  the  articles  that  were  originally  identified  through  the 
literature  review.  For  each  article,  we  attempted  to  obtain  the  following  information:  1)  sample 
size  and  characteristics;  2)  instruments  used;  3)  aims  of  study  (intended  use  of  measures);  4) 
domains  studied;  5)  availability  of  questionnaire  items  within  article;  6)  data  collection  strategies 
employed;  7)  reliability  and  validity  of  data  provided;  8)  casemix  variables;  and  9)  major  study 
findings  (see  Appendix  B). 

We  conducted  over  45  searches.  From  these  searches,  we  reviewed  the  abstracts  of 
articles  that  might  be  relevant  to  PG-CAS  efforts.  A  total  of  230  articles  were  found  from  the  first 
search  strategy  of  which  27  were  selected  for  inclusion  in  our  literature  review.  From  the  second 
search  strategy  we  found  over  400  articles  which  were  narrowed  down  to  87.  From  these  we 
selected  29  articles  for  possible  review.  Of  the  final  56  articles  retrieved  as  “potentially  relevant 
articles”  (i.e.,  articles  that  might  be  useful  to  the  Provider  Group-Consumer  Assessment  Survey 
(PG-CAS)  Instrument  development  effort),  we  found  that  13  articles  found  were  not  applicable 
upon  further  review  because  they  were  marketing  pieces  for  a  managed  care  plan  or  service  (e.g., 
Kirsch,  1969;  Jensen,  1987;  Gaughan,  1993;  Campbell,  1995;  Miller,  1995;  Consumer  Reports, 
1996);  were  theoretical  or  philosophical  pieces  not  deemed  relevant  to  PG-CAS  efforts  (e.g., 

Ware,  Davies-Avery,  and  Stewart,  1978;  Emanual,  1996;  Etter,  1997;  Nelson,  1995;  Sisk,  1990; 
Stange,  1998;  Ware,  Davies,  Rubin,  1987).  We  also  excluded  8  articles  that  did  not  have  any 
components  related  to  physicians  or  physician  groups.  Therefore,  of  the  original  articles  that  were 
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initially  identified  through  our  searches,  we  determined  that  34  were  relevant  to  the  PG-CAS 
effort  because  they  dealt  specifically  with  provider  assessment  or  although  they  focused  on  health 
plans  or  health  systems,  had  instruments  with  questions  related  to  provider  level  assessment  which 
might  be  useful  for  our  instrument  development  efforts.  In  addition,  we  identified  two  other 
unpublished  reports  and  three  articles  published  after  our  searches  were  concluded  which  are  also 
included  in  our  final  set. 
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Table  1: 

Dimensions  of  Patient  Satisfaction  Measured  in  Articles  Reviewed  for  the  PG-CAS  efforts 


Dimensions  studied 

Investigators  (lead  or  sole  author  listed  only) 

Practice  factors 

Accessibility  /  access  /  convenience 

Roberts,  1987;  Merkel,  1984;  Etter,  1997;  Nelson,  1995;  Chung,  1999;  Sixma,  1998;  Safran, 
1998;  Hulka,  1975;  Doyle,  1977;  Wyn,  1997;  Liaw,  1997;  Allen,  1996;  Robbins,  1993; 

Weiss,  1990;  Singh,  1999;  Isenberg,  1998;  Handler,  1998;  Grumbach,  1999;  Grogan,  1995; 

Taira,  1997;  Pacific  Business  Group  on  Health  (1996,  1999);  Hays,  1998;  Morales,  2000; 
Morales,  1999. 

Availability  of  resources 

Roberts,  1987;  Rubin,  1993;  Schmittdiel,  1997;  Doyle,  1977;  Singh,  1999;  Handler,  1998; 

Hays,  1998;  Morales,  2000;  Morales,  1999 

Finances/cost 

Roberts,  1987;  Merkel,  1984;  Etter,  1997;  Safran,  1998;  Hulka,  1975;  Doyle,  1977;  Wyn, 

1997;  Pacific  Business  Group  on  Health  (1996,  1999);  Pacific  Business  Group  on  Health 
(1996,  1999);  Hays,  1998;  Morales,  2000;  Morales,  1999 

Physical  environment 

Merkel,  1984;  Nelson,  1995;  Tessler,  1975;  Doyle,  1977;  Allen  1996;  Weiss,  1990;  Handler, 
1998;  Grogan,  1995;  Hays,  1998;  Morales,  2000;  Morales,  1999 

Efficacy/  outcomes  of  care 

Roberts,  1987;  Bush,  1993;  Allen  1996;  Handler,  1998 

“community  image” 

Nelson,  1995 

Visit  specific  ratings 

Nelson,  1995 

Time  with  physician 

Feletti,  1986;  Liaw,  1997;  Hays,  1998;  Morales,  2000;  Morales,  1999 

“prudence”  [in  terms  of  expenses  and 
risk] 

Doyle,  1977 

Coordination  of  care 

Grumbach,  1999;  Taira,  1997;  Hays,  1998;  Morales,  2000;  Morales,  1999 

Provider  factors: 

Humaneness  /  Art  of  care  (caring, 
empathy,  understanding,  emotional 
aspects) 

Linn,  1984;  Rubin,  1993;  Ross,  1981;  Tessler,  1975;  Chung,  1999;  Poulton,  1996;  Schmittdiel, 
1997;  Roberts,  1987;  DiMatteo,  1980;  Doyle,  1977;  Bush,  1993;  Singh,  1999;  Weingarten, 

1995;  Handler,  1998;  Merkel,  1984;  Thom,  1999;  Sixma,  1998;  Schmittdiel,  1997;  Robbins, 
1993;  Hays,  1998;  Morales,  2000;  Morales,  1999 

Technical  quality  of  care/  Technical 
competence 

Roberts,  1987;  Linn,  1984;  Merkel,  1984;  Rubin,  1993;  Ross,  1981;  Tessler,  1975;  Chung, 

1999;  Thom,  1999;  Schmittdiel,  1997;  DiMatteo,  1980;  Safran,  1998;  Hulka,  1975;  Liaw, 

1997;  Robbins,  1993;  Singh,  1999;  Weingarten,  1995;  Isenberg,  1998;  ;  Taira,  1997;  Pacific 
Business  Group  on  Health  (1996,  1999);  Hays,  1998;  Morales,  2000;  Morales,  1999 

Psychosocial  aspects 

Linn,  1984;  Tessler,  1975;  Schmittdiel,  1997;  Feletti,  1986 

Mutual  participation  by  patient  and 
doctor 

Linn,  1984;  Chung,  1999;  Feletti,  1986 

Interpersonal  aspects  (communication) 

Etter,  1997;  Ross,  1981;  Tessler,  1975;  Chung,  1999;  Poulton,  1996;  Thom,  1999;  Sixma, 

1998;  Sixma,  1998;  Schmittdiel,  1997;  DiMatteo,  1980;  Stewart  1978;  Safran,  1998;  Hulka, 

1975;  Doyle,  1977;  Allen  1996;  Weiss,  1990;  Singh,  1999;  Isenberg,  1998;  Grogan,  1995; 

Pacific  Business  Group  on  Health  (1996,  1999);  Hays,  1998;  Morales,  2000;  Morales,  1999 

Trust 

Thom,  1999;  Safran,  1998; ;  Taira,  1997 

Professional  competence 

Schmittdiel,  1997;  Tessler,  1975;  DiMatteo,  1980;  Feletti,  1986;  Hulka,  1975;  Doyle,  1977; 
Robbins,  1993;  Weiss,  1990;  Singh,  1999;  Isenberg,  1998;  Grogan,  1995;  Taira,  1997;  Hays, 
1998;  Morales,  2000;  Morales,  1999 

Competence  in  emotional  domain 

Feletti,  1986 

Competence  in  social  domain 

Feletti,  1986;  Grogan,  1995 

General  satisfaction 

Roberts,  1987;  Merkel,  1984;  Nelson,  1995;  Schmittdiel,  1997;  Schauffler,  1996;  DiMatteo, 

1980;  Safran,  1998;  Doyle,  1977;  Robbins,  1993;  Isenberg,  1998;  Grumbach,  1999;  Grogan, 

1995;  Pacific  Business  Group  on  Health  (1996,  1999) 

Continuity 

Roberts,  1987;  Poulton,  1996;  Feletti,  1986;  Safran,  1998;  Doyle,  1977;  Allen  1996; 

Grumbach,  1999;  Taira,  1997;  Hays,  1998;  Morales,  2000;  Morales,  1999 

Liaw,  1997 


Expectations/  beliefs/attitudes  about  care 
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Table  2:  Summary  of  Reliability  and  Validity  Information  provided  in  articles  reviewed  for  PG-CAS  efforts  (Continued) 
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Table  2:  Summary  of  Reliability  and  Validity  Information  provided  in  articles  reviewed  for  PG-CAS  efforts  (Continued) 
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Summary  of  Findings 


Among  the  39  articles  that  assessed  physicians  or  group  practices,  27  were  empirical  studies,  6  used 
secondary  data  and  6  were  review  and  or  other  types  of  articles  without  data  analysis.  Studies  that  included 
any  data  collection  relied  on  mail  surveys,  telephone  surveys,  or  self-administered  questionnaires  that  were 
conducted  within  clinics  or  other  health  settings,  in  the  classroom,  or  in  the  community.  Trained  interviewers 
were  used  to  gather  data  in  5  studies  and  data  were  collected  primarily  in  the  clinic  setting  (although  in  2 
studies  a  trained  interviewer  in  the  patient’s  home  collected  data).  Data  collection  methods  could  not  be 
identified  from  information  provided  in  2  articles.  Although  many  of  the  39  studies  that  focused  on  the 
provider  used  multi-item  scales  to  assess  various  aspects  of  patient  evaluations  of  care,  a  few  studies  used 
single-items  to  measure  some  aspect  of  patient  satisfaction  with  provider  or  provider  group  (e.g.,  “Would  you 
refer  your  doctor  to  a  friend?”  “Overall,  how  satisfied  are  you  with  your  provider?”).  Many  studies  used 
“home-cooked”  questionnaires  often  without  providing  reliability  or  validity  information  or  without 
information  on  the  developmental  process  or  how  to  obtain  a  copy  of  the  questionnaire.  As  noted  in  Table  2, 
very  few  articles  from  the  original  56  articles  we  identified  from  our  initial  search  had  any  data  on  reliability  or 
validity  of  the  instruments  used  or  developed.  However,  the  majority  of  the  studies  used  survey  instruments 
that  had  been  used  previously  with  other  populations  (e.g.,  Ware  Patient  Satisfaction  Questionnaire,  MOS 
Visit  Specific  Questionnaire,  Hulka  &  Zyzanski  questionnaire,  GHAA  Patient  Satisfaction  Questionnaire). 

Instruments  Used  Multiple  Times 

Among  the  33  articles  that  included  any  data  analysis  (27  from  primary  and  6  from  secondary  data 
sources),  we  found  that  29  used  items  or  instruments  that  had  been  previously  developed  and  that  some 
adapted  them  for  the  specific  purpose  of  their  study  (e.g.,  Ware’s  Patient  Satisfaction  Questionnaire,  Hulka  & 
Zyzanski’s  satisfaction  questionnaire,  items  from  the  Medical  Outcomes  Study  Visit  Specific  Questionnaire). 
Others  relied  on  items  collected  in  large  scale  or  national  studies  (e.g..  Health  Plan  Value  Check, 
Commonwealth  Fund’s  1994  Managed  Care  Survey,  or  the  Dutch  National  System  Survey).  In  the  six  studies 
that  focused  on  provider  assessment,  the  authors  developed  their  own  surveys  or  instruments  using  various 
methodologies  to  assess  patient  satisfaction  with  general  practitioners  or  other  providers.  Only  two  of  these 
studies,  however,  presented  psychometric  properties  of  the  instrument.  The  majority  of  the  studies  used  multi¬ 
item  scales  to  measure  a  given  dimension  of  patient  satisfaction  while  a  few  studies  (particularly  those  that 
developed  their  own  survey)  used  a  single-item  question  to  assess  patient  satisfaction  with  their  provider  or 
group  practice.  Overall,  for  all  the  reviewed  articles  considered  relevant  to  the  PG-CAS,  we  found  that  26  of 
the  39  articles  used  previously  developed  instruments  while  13  studies  did  not. 

In  the  following  section,  we  provide  an  expanded  summary  of  the  most  frequently  used  instruments. 
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Patient  Satisfaction  Questionnaire 


We  identified  several  studies  that  used  a  version  of  the  Patient  Satisfaction  Questionnaire  (PSQ). 

Some  used  the  entire  PSQ,  others  used  certain  sub-scales,  and  still  others  adapted  questions.  The  PSQ  was 
used  to  assess  care  within  various  outpatient  settings  and  in  a  community  setting.  The  study  groups  in  which 
the  PSQ  was  used  included,  for  example,  community  members,  Medicaid  recipients,  general  patient 
populations  seen  in  family  practice  and  general  practice  clinics,  and  patients  discharged  after  hospitalization 
for  myocardial  infarction. 

The  earliest  study  we  identified  for  this  literature  review  which  used  the  PSQ  (Doyle  &  Ware,  1977) 
assessed  the  importance  of  consumer  perceptions  of  various  characteristics  of  physicians  and  medical  care 
services  in  relation  to  general  satisfaction  with  medical  care  and  determined  whether  each  characteristic  had 
equal  importance  across  selected  groups  differing  in  demographic  and  socioeconomic  characteristics. 
Respondents  completed  a  32-page  self-administered  questionnaire  that  included  the  51 -item  PSQ  and 
questions  about  sociodemographic  information,  general  health  perceptions,  use  of  health  services,  and  general 
satisfaction  with  medical  care.  A  mixed  sampling  design  (cluster,  regular  interval,  and  proportional  stratified 
sampling)  was  used  to  draw  520  households  with  432  completing  the  survey.  Internal  consistency  reliability 
coefficients  ranged  from  0.50  to  0.91  with  a  median  of  0.71  for  the  17  scales.  Six  major  dimensions  of 
consumer  perceptions  regarding  physicians  and  their  medical  care  were  identified  using  factor  analysis  of 
survey  data:  physician  conduct,  continuity  of  care,  accessibility,  availability  of  hospitals/specialists, 
completeness  of  facilities,  and  availability  of  family  doctors.  Scores  for  these  dimensions  and  multivariate 
statistical  methods  were  used  to  predict  general  satisfaction  ratings  for  a  cross  section  of  adults  and  for  groups 
differing  in  age,  education,  health  status,  and  sex.  Five  of  these  components  were  significantly  related  to 
general  satisfaction,  with  physician  conduct  having  the  strongest  association  (accounting  for  41%  of  the 
variance). 

A  later  study  conducted  by  Merkel  and  colleagues  (1984)  examined  the  relationship  between 
physicians’  expectations  of  patient  satisfaction  and  actual  patient  satisfaction  among  22  patients  within  a 
university-affiliated  family  health  center  using  the  short  form  of  PSQ-II.  The  items  in  the  patient’s  general 
satisfaction  scale  were:  1)  “I’m  very  satisfied  with  the  medical  care  I  receive,”  2)  “The  care  I  have  received 
from  doctors  is  just  about  perfect,”  3)  “Most  people  receive  medical  care  that  could  be  better,”  and  4) 
“Doctors  respect  their  patient’s  feelings.”  Physicians  completed  the  same  questionnaire  prior  to  the  study, 
with  instructions  to  answer  how  they  thought  the  typical/average  patient  would  respond  and  also  completed 
the  Social  Factors  in  Illness  and  the  Doctor-Patient  Relations  subscales  of  the  Attitudes  toward  Social  Issues 
in  Medicine  Questionnaire.  In  addition,  physicians  completed  a  three-item  questionnaire  immediately  after 
seeing  each  patient,  which  assessed  the  physician’s  perception  of  patient  satisfaction  with  the  visit,  the 
physician’s  own  satisfaction  with  medical  care  rendered,  and  the  physician’s  liking  for  the  patient.  Patients 
completed  the  patient  questionnaire  after  their  visit.  In  this  study,  patient  satisfaction  was  significantly  and 
positively  correlated  with  how  much  the  physician  liked  the  patient.  The  major  finding  was  that  advanced 
family  practice  residents  could  not  accurately  predict  their  patients’  level  of  satisfaction  with  medical  care. 

In  another  study,  the  authors  compared  two  patient  satisfaction  questionnaires  (Hulka  instrument  with 
the  PSQ)  and  assessed  the  quality  of  the  data  collected  (Roberts  and  Tugwell,  1987).  Patients  hospitalized  for 
myocardial  infarction  were  asked  to  complete  both  instruments  following  an  outpatient  visit  (at  4  and  again  at 
6  months  after  discharge).  Order  of  questionnaire  completion  was  counterbalanced.  The  Hulka  questionnaire 
measured  satisfaction  in  three  domains:  professional  competence,  personal  qualities,  and  cost/convenience. 
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The  Ware  questionnaire  assessed  satisfaction  in  8  areas:  access/convenience,  finances,  availability  of 
resources,  continuity  of  care,  quality/competence  of  physician,  humaneness  of  physician,  general  satisfaction, 
and  efficacy  of  care.  A  total  of  67  patients  were  recruited  into  the  study  (59  patients  participated  in  the  month 
4  survey,  43  in  the  six-month  survey).  Overall,  there  were  few  dissatisfied  patients  (0-7  percent).  Likert  mean 
scores  were  determined  for  each  questionnaire  and  found  that  they  gave  similar  results.  The  two 
questionnaires  were  highly  correlated  using  Pearson  correlation  coefficients  for  total  scores  (4  months, 
r=0.81;  6  months,  r=0.75).  Components  that  had  the  highest  mean  scores  in  both  the  Hulka  and  PSQ  dealt 
with  the  physician’s  personal  qualities  and  professional  competence.  In  the  PSQ,  patients  expressed  lower 
satisfaction  with  finances  and  greater  satisfaction  with  continuity  and  humaneness  of  physician.  Pearson 
correlation  coefficients  for  four-  and  six-month  Likert  scores  were  calculated  with  43  patients  who  completed 
surveys  at  both  times:  PSQ  (r  =  .64  (90%  Cl  =  .46-  .77)  and  Hulka  questionnaire  (r  =  .59,  90%  Cl  =  .40  - 
.74). 


Roberts  and  Tugwell  (1987)  reclassified  the  PSQ  statements  according  to  the  three  Hulka  component 
headings:  1)  PSQ  subscales  on  quality/competence  of  the  physician,  general  satisfaction,  and  efficacy  of  care 
were  reclassified  into  Hulka’ s  professional  competence  scale  (PC);  2)  PSQ  humaneness  of  physician  subscale 
was  reclassified  into  Hulka’s  personal  qualities  scale  (PQ);  and  3)  PSQ  subscales  on  access/convenience, 
finances,  available  resources  and  continuity  of  care  were  reclassified  into  Hulka’s  cost/convenience  (CC). 
Comparable  results  were  found  in  both  the  Hulka  and  revised  PSQ  questionnaires.  The  Pearson  correlation 
coefficients  for  the  PC,  PQ  and  CC  component  headings  were  r  =.56,  r  =  .65,  and  r  =  .52,  respectively.  The 
PSQ  had  an  1 1 -point  higher  total  score  compared  to  the  Hulka  questionnaire  (paired  t-test,  p  <  .001). 

Robbins,  Bertakis,  Helms,  and  colleagues  (1993),  assessed  the  influence  of  physician  practice 
behaviors  on  patient  satisfaction  among  100  patients  seen  at  a  university-based  residency  teaching  clinic  using 
physician  interviews,  observations  of  doctor-patient  encounters  (scored  using  the  Davis  Observation  Code, 
DOC)  and  patient  interviews.  Patients  completed  a  pre-visit  questionnaire  (using  the  original  PSQ)  and  a 
revised  post-visit  questionnaire  that  assessed  patient  satisfaction  with  the  specific  encounter  and  provider.  The 
encounter  was  videotaped  and  physician  behavior  scored/characterized  using  the  DOC.  The  following 
components  of  patient  satisfaction  were  assessed  in  an  18-question  survey:  general  satisfaction,  physician 
humaneness,  quality/competence  and  access  to  care.  Questions  were  summed  to  come  up  with  a  total  visit- 
specific  satisfaction  score.  Using  multiple,  stepwise  regression  analysis,  total  visit-specific  satisfaction  was 
significantly  related  to  total  pre-visit  satisfaction  and  to  four  DOC  variables:  health  education,  physical 
examination,  the  discussion  of  treatment  effects,  and  history  taking.  The  general,  humaneness,  and 
quality/competence  subscales  were  positively  related  to  the  first  three  DOC  variables  but  negatively  related  to 
history  taking. 

Allen,  Stoline,  Yang  and  Barrett  (1996)  used  an  adapted  version  of  the  PSQ  in  a  cross-sectional  study 
of  436  Medicaid  and  non-Medicaid  patients  attending  four  family  practice  residency  clinics.  The  39-item 
survey  included  questions  on  demographics  and  five  dimensions  of  patient  satisfaction  (interpersonal  manner, 
efficacy/outcomes,  accessibility/convenience,  continuity  and  physical  environment).  It  also  had  a  composite 
score  for  life  satisfaction,  one  measure  of  patient  confidence  in  the  local  medical  care  community,  and  also 
included  questions  about  how  patients  felt  they  were  treated  based  on  insurance  type.  The  authors  did  not  find 
any  evidence  that  Medicaid  enrollees  were  less  satisfied  with  their  medical  care  than  other  patients  in  family 
practice  residency  clinics  after  adjusting  for  the  effects  of  life  satisfaction,  confidence  in  the  local  medical  care 
community,  and  various  demographic  factors. 
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More  recently,  Handler,  Rosenberg,  Raube,  and  Kelley  (1998)  adapted  a  version  of  the  PSQ  to 
explore  the  relation  between  characteristics  of  prenatal  care  and  satisfaction  among  75  African  American  and 
26  Mexican  American  Medicaid  recipients  who  participated  in  a  25-minute  telephone  interview.  The  survey 
had  questions  related  to  satisfaction  with  prenatal  care  (art  of  care,  technical  quality,  physical  environment, 
access,  availability  and  efficacy),  prenatal  care  characteristics  (practitioner  attributes,  service  availability,  and 
features  of  delivery  of  care)  and  personal  characteristics.  The  validity  of  the  satisfaction  scale  was  measured 
by  the  correlation  observed  between  the  overall  scale  and  two  questions  related  to  whether  they  would 
recommend  the  site  to  a  friend  who  needed  prenatal  care  and  their  overall  rating  of  their  overall  care. 
Cronbach’s  alpha  coefficients  were  reported  for  the  overall  scale  (a  sum  of  all  items  in  the  five  sub-scales  and 
one  question  regarding  efficacy  of  care)  and  specific  satisfaction  scales:  overall  satisfaction  (a=  0.95),  art  of 
care  (a=  0.91),  technical  quality  (a=  0.81),  physical  environment  (a=  0.78),  access  (cc=  0.73),  and  availability 
(a=  0.56).  Having  procedures  explained  by  the  provider,  short  waiting  times,  access  to  ancillary  services,  and 
having  a  male  prenatal  care  practitioner  were  all  associated  with  increased  satisfaction  scores. 

Medical  Outcomes  Study  (MOS)  Visit  Specific  Questionnaire  (VSO) 

Nine  articles  used  the  Medical  Outcomes  Study  (MOS)  Visit  Specific  Questionnaire  (VSQ)  to  assess 
perceptions  of  a  specific  ambulatory  visit.  Rubin  and  colleagues  (1993)  conducted  analyses  of  data  from  the 
Medical  Outcomes  Study  (MOS)  to  assess  17,671  patient’s  ratings  of  specific  outpatient  visits  across  five 
systems  of  care  (combinations  of  solo/multi-specialty  group/HMO  practices  and  fee-for-service/prepaid)  in 
several  large,  metropolitan  areas.  The  MOS  VSQ  includes  a  global  rating  of  the  visit  and  eight  items  assessing 
the  quality  of  specific  features  of  the  visit  and  access  to  care  (technical  skills,  personal  manner,  appointment 
wait  time,  office  wait  time,  time  spent  with  the  provider,  and  what  was  done  at  the  visit)  using  an  excellent  to 
poor  response  scale.  Results  were  weighted  for  sampling  probability  and  clustering  effects.  Most  patients 
rated  their  physicians’  technical  skills  and  personal  manner  as  excellent  (64%  and  73%,  respectively). 
Explanations  of  care  and  appointment  waits  were  rated  as  excellent  by  fewer  patients  (56%  and  53%, 
respectively).  Patients  of  solo  practices  rated  all  aspects  of  their  care  better  than  HMO  patients  did,  most 
markedly  appointment  waits,  and  telephone  access.  Adjusting  for  patients’  demographics,  diagnoses  and  self- 
rated  health  did  not  change  results.  Physicians  with  visit  ratings  in  the  lowest  20%  were  nearly  four  times  as 
likely  to  be  left  by  patients  within  6  months  than  physicians  in  the  highest  20%. 


Schmittdiel,  Selby,  Grumbach,  and  Quesenberry  (1997)  conducted  a  cross-sectional  mail  survey  to 
compare  the  satisfaction  of  care  between  members  of  a  group  health  maintenance  organization  who  chose 
their  primary  care  provider  and  members  who  were  assigned  a  physician.  A  total  of  16,109  patients  from  the 
practices  of  60  family  physicians,  245  general  internists,  and  55  subspecialists  who  were  part  of  the  Kaiser 
Permanente  Medical  Care  Program  of  Northern  California  were  sent  a  survey  (total  respondents=  10,205). 
Patients  who  chose  their  personal  physician  (N=4,748)  consistently  reported  higher  satisfaction  scores 
{excellent  or  very  good)  on  the  9  MOS  VSQ  items  compared  to  patients  who  were  assigned  a  physician 
(N=5,457)  (p  <  .001  for  each  comparison).  Patients  who  chose  or  were  assigned  a  physician  were  very 
similar  in  how  they  valued  most  aspects  of  physician  care.  However,  for  two  items,  (‘The  physician  spends 
sufficient  time  with  patient”,  and  ‘The  physician’s  personal  manner  is  courteous,  respectful,  sensitive  and 
friendly”)  patients  who  chose  their  physician  were  slightly  more  likely  to  value  these  as  “more  important  than 
anything  else.”  In  a  logistic  regression  model  that  adjusted  for  various  covariates,  having  chosen  one’s 
physician  was  the  predictor  most  strongly  related  to  rating  the  overall  visit  to  be  excellent  (OR  2.18,  Cl  1.95- 
2.42).  Other  significant  predictors  of  higher  satisfaction  were  older  age,  being  male,  having  lower  income, 
having  better  physical  and  mental  health  scores,  the  physician’s  “popularity”,  and  valuing  a  physician’s 
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personal  manner.  In  further  analyses,  however,  the  association  of  provider  choice  and  satisfaction  was  similar 
in  men  and  women  and  did  not  differ  by  patient’s  age,  race,  income,  or  level  of  education. 

Isenberg  and  Stewart  (1998)  adapted  the  MOS  VSQ  in  a  study  of  community-based  medical  practices. 
The  VSQ  was  used  to  examine  the  influence  of  patients’  perception  of  quality  on  physicians’  initiatives  to 
improve  that  perception.  Overall,  6088  patients  from  59  physician  practice  offices  participated  (n=3815  from 
29  physician  offices  in  control  group  and  n=2273  patients  from  30  physicians  in  the  intervention  group).  After 
the  baseline  surveys  were  completed,  all  participating  sites  were  sent  their  benchmark  scores  and  the 
intervention  sites  received  a  Quality  Improvement  Process  (QIP)  poster  to  display  in  visible  locations  of 
office.  One  month  after  they  were  sent  a  2nd  round  of  questionnaires  for  patients  to  complete  (16  of  30  sites 
returned  at  least  30  patient  questionnaires).  The  proportion  of  excellent  responses  on  the  VSQ  was  calculated 
for  each  item.  Next,  summary  scores  for  the  “access”  and  “physician  attributes”  sub-components  were 
calculated  as  the  mean  score  of  the  individual  items  making  up  the  summary  score.  The  change  in  proportion 
of  excellent  responses  was  calculated  for  both  the  intervention  and  control  group.  The  intervention  group 
demonstrated  statistically  significant  improvements  in  access  (4.2%  increase,  p  =  0.02),  physician  attributes 
(5.7%  increase,  p  <  .001),  and  overall  visit  scores  (5.5%  increase,  p  =  .002)  compared  to  the  control  group. 
The  authors  analyzed  the  results  by  individual  items  and  summary  scores  and  found  that  the  differences 
between  the  two  groups  remained  significant  for  three  of  the  four  items  of  the  physician  attributes  scale 
(explanation,  time  with  doctor,  and  interpersonal  quality  of  care). 

In  another  study  (Chung,  1999),  345  consecutive  patients  seen  in  an  outpatient  plastic  surgery  clinic 
filled  out  a  questionnaire,  which  included  the  9  MOS  VSQ  items.  Using  multiple  logistic  regression  analysis, 
the  author  identified  four  variables  that  were  significant  predictors  of  the  overall  visit  rating:  1)  personal 
manner  of  the  physician;  2)  time  spent  with  physician;  3)  length  of  time  to  get  an  appointment;  and  4) 
explanation  of  what  was  done.  None  of  several  possible  casemix  variables  (age,  gender,  ethnicity,  and 
education)  were  significantly  associated  with  the  overall  rating  of  the  visit. 

Grumbach,  Selby,  Smittdiel,  and  Quesenberry  (1999)  conducted  a  cross-sectional  mail  survey  in  1995 
using  items  from  the  MOS  VSQ  and  other  questions  to  determine  if  physician  specialty  was  associated  with 
differences  in  the  quality  of  primary  care  styles  and  patient  satisfaction  in  a  large,  group  model  HMO.  A 
stratified,  random  sample  of  patients  who  met  entry  criteria  were  selected  in  such  a  way  that  approximately 
equal  numbers  of  patients  were  sampled  from  each  of  three  main  physician  specialty  groups.  The  final  sample 
consisted  of  10,608  patients,  ages  35-85  years,  from  the  practices  of  60  Family  practitioners  (FP),  245 
General  internists  (GIM)  and  55  subspecialty  internists  (SIM).  Health  status  was  measured  with  the  SF-12, 
and  primary  care  attributes  assessed  included  items  on  coordination,  comprehensiveness,  accessibility, 
preventive  care  procedures,  health  promotion — lifestyle  and  social  components  (adapted  from  Starfield,  Brice, 
Schach,  Rabin,  et  al,  1973).  No  significant  differences  were  found  among  patients  in  the  three  physician 
groups  on  health  values  and  beliefs,  ratings  of  the  quality  of  primary  care,  and  satisfaction.  Overall,  all  three- 
physician  specialty  groups  had  comparable  and  high  overall  satisfaction  ratings.  Patients  rated  GIMs  higher 
than  FPs  on  coordination  of  care,  (adjusted  mean  scores  68.0  and  58.4,  respectively,  p  <  .001)  and  slightly 
higher  on  accessibility  (50.9  and  46.7,  respectively,  p  <  .01)  and  prevention  (86.6  and  84.1,  respectively,  p  < 
.05).  GIMs  were  rated  more  highly  than  SIMs  on  comprehensiveness  (76.4  and  73.8,  respectively,  p  <  .01). 

No  significant  differences  were  found  on  measure  of  patient  satisfaction  among  the  three  physician  groups. 
Primary  care  practice  did  not  differ  substantially  across  physician  specialty  within  this  group  model  HMO. 

The  previous  studies  all  used  MOS  to  assess  quality  of  care  at  the  health  plan  level  but  did  not  analyze 
data  at  the  physician  group  level.  We  identified  one  study  (Hays,  Brown,  Spritzer,  Dixon  and  Brook,  1998) 
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that  examined  the  data  obtained  through  the  Medical  Quality  Commission  to  assess  the  HMO  care  provided 
by  the  Unified  Medical  Group  Association  which  included  data  on  63  multispecialty  medical  groups.  This  was 
one  of  the  first  attempts  by  the  Medical  Quality  Commission  to  conduct  a  standardized  member  study  to 
compare  performance  among  physician  groups.  Data  from  the  MOS  36-item  short  questionnaire  was  analyzed 
to  present  ratings  of  the  care  provided  by  physician  groups  to  enrollees  in  a  variety  of  capitated  maintenance 
organization  plans.  A  random  sample  of  adult  enrollees  receiving  managed  health  care  from  48  physician 
groups  was  sent  a  12  page  questionnaire  which  included  153  items  that  assessed  49  health  care  rating  items  in 
the  following  areas:  intention  to  switch  to  another  physician  group;  intention  to  switch  to  another  health  plan; 
ratings  of  care;  reports  about  care;  utilization;  health  status;  comorbidities;  and  demographics.  The  majority 
of  the  questions  were  administered  using  a  7-point  response  scale  (very  poor,  poor,  fair,  good,  very  good, 
excellent,  and  the  best)  along  with  an  option  for  “does  not  apply  to  me.”  A  total  of  7093  questionnaires  were 
returned  (59%  response  rate).  Six  scales  were  constructed  for  the  following  domains:  1)  overall  quality  of 
care  provided  by  the  physician  group;  2)  quality  and  convenience  of  care  provided;  3)  access  to  care;  4)  office 
waiting  time;  5)  choice  of  primary  care  provider;  and  6)  coverage  for  mental  health  care.  Each  scale  was 
scored  by  averaging  the  items  on  the  scale.  Internal  consistency  reliability  for  the  multi-item  scales  were 
excellent  (the  alpha  coefficients  were  .81  or  higher  for  7  of  the  8  SF-36  scales.  And  ranged  from  0.77  to  0.98 
for  the  6  health  care  rating  scales).  They  found  clinically  and  statistically  significant  differences  in  ratings 
among  the  various  groups.  Negative  ratings  of  care  were  significantly  related  to  the  following:  intention  to 
switch  to  another  physician  group,  difficulties  getting  appointments,  lengthy  waiting  periods  in  the  reception 
area  and  waiting  room,  the  inability  to  get  consistent  care  from  one  physician  for  routine  visits,  and  not  being 
informed  by  the  office  staff  when  there  was  a  delay  in  seeing  the  primary  care  provider. 

In  the  second  study,  Morales  and  colleagues  (1999)  also  used  data  from  the  Unified  Medical  Group 
Association  which  included  data  on  63  multispecialty  medical  groups.  In  their  study,  they  examined  the 
association  of  patient  ratings  of  communication  by  providers  with  ethnicity  (Latino/  white)  and  language 
(Spanish/  English).  Data  were  analyzed  for  6,21 1  surveys  where  the  respondents  were  Spanish  speaking 
Latinos  (Latino/  Spanish)  English-speaking  Latinos  (Latino/English)  or  white.  The  authors  assessed  patient 
satisfaction  with  provider  communication  in  five  areas:  medical  staff  listening  to  what  they  had  to  say;  answers 
to  their  questions;  explanations  about  prescribed  medications;  explanations  about  medical  tests;  and 
reassurance  and  support  from  their  doctor  and  support  staff.  Differences  in  demographic,  socioeconomic,  and 
health  status  characteristics  among  Latino/Spanish,  Latino  English,  and  white  respondents  were  examined 
using  bivariate  analyses.  A  communication  summary  score  was  constructed  by  averaging  the  five  provider 
communication  questions.  Then  the  score  was  normalized  to  a  mean  of  50  and  SD  of  10  (T  score). 
Associations  between  this  T  score  and  each  independent  variable  were  examined.  Then  each  satisfaction-with- 
communication  question  was  modeled  using  multivariate  logistic  regression.  The  authors  found  that  Spanish 
speaking  Latinos  (Latino/Spanish)  were  significantly  more  dissatisfied  compared  with  Latinos  responding  in 
English  (Latino/  English)  and  non-Latino  whites  on  all  5  communication  areas. 


Other  Survey  Instruments 

As  mentioned  previously,  of  the  33  articles  that  had  any  data  presented,  in  6  of  the  studies  relating 
specifically  to  providers,  the  authors  developed  their  own  questionnaires  to  assess  patient  satisfaction  (Ross  et 
al.,  1981;  Tessler  and  Mechanic,  1975;  Feletti,  Firman,  &  Sanson-Fisher,  1986;  DiMatteo  &  Hays,  1980; 
Hulka,  et  al.,  1975)  and  investigate  its  relations  to  various  patient-,  provider-,  or  practice-specific 
characteristics.  Additional  articles  that  were  primarily  focused  at  the  health  plan  level  but  also  had  questions 
related  to  the  provider  are  included  here  as  well.  The  following  domains  were  assessed  in  these  instruments: 
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•  Technical  /  professional  competence  of  the  physician  (Ross  et  al.,  1981;  Hulka  et  al.,  1975; 
DiMatteo  &  Hays,  1980;  Feletti  et  al,  1986) 

•  Emotional  /  social  competence  of  the  physician  (Tessler  and  Mechanic,  1975;  DiMatteo  &  Hays  , 
1980;  Feletti  et  al.,  1986). 

•  Interpersonal  characteristics  of  the  physician  (e.g.,  friendliness,  openness,  respectful,  courteous, 
warmth,  personal  interest  in  the  patient)  (Hulka,  et  al.,  1975;  DiMatteo  &  Hays,  1980;  Feletti  et 
al.,  1986;  Hays,  et  al.,  1998;  Morales,  et  al.,  1999)  or  staff  members  (Tessler  and  Mechanic,  1975). 

•  Doctor-patient  communication  (e.g.,  doctor  listens  to  patients,  communicates  information  to 
patients)  (Tessler  and  Mechanic,  1975;  Hulka,  et  al.,  1975;  Singh,  Haqq,  Mustapha,  1999; 
DiMatteo  &  Hays,  1980;  Feletti,  et  al.,  1986.) 

•  Access  to  care  (cost,  convenience)  (Hulka,  et  al.,  1975;  Singh,  et  al.,  1999). 

•  Continuity  of  care  (Feletti  et  al.,  1986). 

•  Mutual  understanding  between  provider  and  patient  (Feletti  et  al.,  1986). 

•  Attitudes  and  expectations  about  care  (Singh,  et  al.,  1999;  DiMatteo  &  Hays,  1980). 


Hulka  and  colleagues  (1975)  developed  a  questionnaire  using  Thurstone  scaling  methods  in 
conjunction  with  a  Likert  format  and  a  modified  scoring  technique  that  incorporated  both  the  Thurston  scale 
values  and  the  specific  weightings  determined  by  the  response  alternatives  selected  by  the  respondents.  The 
scale  was  designed  to  measure  the  respondents’  attitudes  with  respect  to:  a)  the  professional  and  technical 
competence  of  the  physician;  b)  personal  qualities  of  the  physician  in  his  relationship  with  the  patient;  and  c) 
accessibility  of  care,  including  cost  and  convenience.  In  this  community-wide  sample  of  adult  respondents, 
attitudes  toward  physicians  and  medical  care  tended  to  be  favorable.  People  were  least  satisfied  with  cost  and 
convenience  of  care  as  compared  with  physicians’  professional  competence  or  personal  qualities  in  the  doctor- 
patient  relationship.  Age,  sex,  race,  marital  status,  family  size,  education,  occupation,  social  class,  place  of 
birth,  and  duration  of  residence  were  cross-tabulated  against  overall  and  specific  satisfaction  scores.  There 
were  marked  and  consistent  associations  among  race,  sex  and  the  percentage  of  persons  satisfied.  Females 
were  less  satisfied  than  males  and  blacks  had  lower  proportion  of  high  satisfaction  scores  than  whites.  Older 
respondents  (>  60)  had  the  lowest  percent  of  high  satisfaction  scores.  Persons  in  the  lower  social  class  were 
least  satisfied  with  all  aspects  of  medical  care.  Having  a  regular  physician  and  long  attendance  with  that 
physician  were  variables  associated  with  higher  satisfaction. 

In  another  study  (Tessler  and  Mechanic,  1975),  the  authors  used  an  1  l-item  measure  that  measured 
the  following:  amount  of  privacy  in  doctor’s  office;  amount  of  time  doctor  spends  with  you;  doctor’s  concern 
about  your  health;  doctor’s  warmth  and  personal  interest  in  you;  amount  of  information  given  to  you  about 
your  health;  doctor’s  training  and  technical  competence;  doctor’s  friendliness;  friendliness  of  nurses, 
receptionists,  etc.;  quality  of  medical  care  received;  adequacy  of  office  facilities  and  equipment;  and  doctor’s 
willingness  to  listen  when  you  tell  him  about  your  health.  Each  item  was  answered  as  3  =  “very  satisfied,  “  2  = 
“somewhat  satisfied,  ”  and  1  =  “not  satisfied  at  all.”  They  were  also  asked  items  related  to  their  value  of  and 
belief  in  medical  care.  In  addition  to  asking  about  satisfaction  with  their  own  care,  respondents  were  asked 
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about  satisfaction  with  medical  care  received  by  their  children.  Those  in  prepaid  group  practice  (n  =  356) 
were  significantly  less  satisfied  than  those  in  Blue  Cross  (n  =  354)  on  many  aspects  of  their  care  and  most 
specifically  with  the  professional/technical  competence  of  their  physicians,  the  interpersonal  communication 
with  their  doctors,  and  the  adequacy  of  the  facilities  and  equipment.  The  authors  developed  satisfaction 
indices  for  respondent’s  satisfaction  with  own  medical  care  and  with  care  of  their  children.  For  the  satisfaction 
index,  the  scores  of  each  individual  item  were  added  to  form  a  summary  score.  Possible  ranges  of  scores  were 
1 1-33.  Index  of  satisfaction  with  own  care  had  a  mean  score  of  30.48  (S.D.  =  4.05)  and  satisfaction  with  care 
received  by  their  children  was  30.76  (SD  =  4.21).  Blue  Cross  participants  were  less  satisfied  in  general 
compared  to  participants  in  prepaid  plan.  Those  with  greater  skepticism  toward  medical  care  (example  item: 
“Do  you  often  doubt  some  of  the  things  doctors  say  they  can  do?”)  and  less  faith  in  doctors  (e.g.,  item 
required  choosing  between  “I  have  great  faith  in  doctors”;  “In  general,  I  think  doctors  do  a  good  job”;  “In 
general,  I  think  most  doctors  are  over-rated”;  “I  distrust  doctors”)  were  significantly  less  satisfied  with  care. 

DiMatteo  and  Hays  (1980)  studied  the  significance  of  patients’  perceptions  of  physician  conduct 
within  a  family  practice  center  and  examined  the  interrelationships  among  various  aspects  of  patients’ 
perceptions  of  their  physician’s  communication  ability,  affective  behavior  and  technical  competence  and  the 
association  of  each  domain  to  overall  satisfaction  with  the  physicians.  They  developed  four  scales:  patient’s 
general  satisfaction  with  and  commitment  to  their  physician;  their  perception  of  the  physician  as  having 
communicated  sufficient  information  to  them;  their  perception  of  the  physician’s  affective  behavior;  and  their 
physician’s  technical  competence.  Three  of  the  four  dimensions  of  physician  behavior  correlated  highly  with 
each  other — communication,  affective  care,  and  technical  competence.  The  entire  25  item  scale  had  high 
internal  consistency  reliability  (Cronbach’s  a  =  0.92)  and  moderate  test-retest  reliability  (r  =  0.63,  n  =  22). 
There  were  no  significant  correlations  between  the  socioeconomic  status  of  patients  and  their  general 
satisfaction  with  care  or  their  perceptions  of  various  aspects  of  physician  conduct.  Patients’  satisfaction  with 
their  physicians  was  highly  related  to  their  perceptions  of  the  technical  quality  of  care  as  well  as  their  feelings 
that  the  physician  communicated  with  them  as  well  as  cared  about  them  as  people. 

Ross  et  al.  (1981)  studied  the  primary  care  provided  to  372  children  of  61  pediatricians  in  New  Haven 
(Connecticut)  and  assessed  the  impact  of  provider  behavior  on  patient  satisfaction  using  the  following  items: 
a)  “If  a  close  friend  just  had  her  first  baby  and  asked  you  for  your  advice  about  pediatricians,  would  you 
recommend  your  pediatrician?”  b)  “Have  you  ever  considered  changing  pediatricians?”;  c)  “Is  your 
pediatrician  available  to  you  when  you  need  him/her?”;  d)  “What  is  your  opinion  of  your  pediatrician?”;  e) 
“Have  you  ever  found  his/her  advice  confusing  or  harmful?”;  f)  “Do  you  usually  follow  his/her  advice?”;  and 
g)  “How  does  your  pediatrician  compare  with  the  ideal  pediatrician?”  These  items  were  used  to  develop  one 
general  satisfaction  measure.  In  addition,  the  authors  observed  the  encounter  between  the  patient  and  doctor 
and  interviewed  the  physician.  The  physicians  were  rated  by  the  observers  on  technical  quality  (history-taking, 
physical  exam,  laboratory  utilization,  diagnosis,  treatment,  prescription  writing,  follow-up)  and  psychosocial 
care  (how  well  the  physician  knows  the  family,  whether  the  physician  listens  to  the  client,  respects  his  or  her 
wishes,  and  fully  explains  the  illness  and  the  treatment  to  the  client).  The  psychosocial  component  of  care  had 
the  largest  association  with  client  satisfaction  whereas  technical  quality  was  not  significantly  related  to 
satisfaction.  The  number  of  minutes  the  physician  spent  with  the  patient  in  the  visit,  and  the  child’s  health 
were  not  significantly  associated  with  general  satisfaction. 

Singh,  Haqq,  and  Mustapha  (1999)  assessed  the  following  for  doctors,  nurses  and  pharmacists: 
satisfaction  with  the  courtesy  and  consideration  shown;  satisfaction  with  the  medical  skills  and  competence; 
willingness  to  listen  to  patients’  explanations  of  problems;  satisfaction  with  advice  given;  waiting  time,  and 
duration  of  examination  (responses  available  were  “satisfied”  “undecided”  or  “dissatisfied”).  Approximately 
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74  percent  of  interviewees  were  “satisfied”  with  their  doctor  although  more  were  satisfied  with  the  nurses. 

The  greatest  needs  for  improvement  were  related  to  the  doctor’s  and  pharmacist’s  services  and  waiting  time. 
No  data  on  reliability  or  validity  were  presented. 

Four  studies  had  as  one  of  their  main  goals  the  development  and  validation  of  a  patient  satisfaction 
instrument.  Feletti,  Firman,  and  Sanson-Fisher  (1986)  examined  patients’  perceptions  of  their  latest 
consultation  with  their  primary  care  physician.  The  questionnaire  had  items  related  to  the  following  10  areas: 
competence  in  a  physical  domain,  competence  in  the  emotional  domain,  competence-social  awareness, 
physician  as  a  model  (was  GP  seen  by  patients  to  be  modeling  a  desirable  style  of  living),  amount  of  time  for 
consultation,  perceive  amount  of  continuity  of  care  by  the  physician,  mutual  understanding  in  the  doctor- 
patient  relationship,  patients’  perception  of  their  individuality,  information  transfer,  and  competence-physical 
domain.  Patients  who  saw  one  of  22  doctors  in  various  doctors’  offices  were  asked  to  complete  the  49-item 
“ideal  physician”  survey  (which  included  43  questions  related  to  an  ideal  physician  and  6  additional  questions 
on  their  health  beliefs)  prior  to  their  consultation  with  their  doctor  and  were  also  asked  to  complete  a 
questionnaire  describing  their  actual  encounter.  Each  item  on  the  49-item  ideal  physician  questionnaire  used  a 
Likert  6-point  scale  (strongly  agree  to  strongly  disagree  with  NA  response  category  also  available).  Responses 
provided  by  503  patients  on  the  “ideal  physician”  questionnaire  had  high  internal  consistency  with  an  alpha 
coefficient  of  0.91.  In  general,  patients  appeared  quite  satisfied  with  the  behavior  of  their  physicians  (mean 
2.31  ±  0.6)  compared  to  their  rating  of  the  “ideal”  physician  (1.7  ±  0.43).  Principal  components  analysis 
revealed  five  components  that  accounted  for  40%  of  the  variance:  a)  communication,  care  and  reassurance 
(24%);  b)  professional  attitudes  and  behaviors  (5%);  c)  personal  confidant  of  patient  (5%);  d)  technical 
competence  (4%);  and  e)  generating  trust  in  physician  (3%).  Following  this  analysis,  a  24-item  reduced 
survey  was  created  by  selecting  items  with  strong  loadings  on  the  first  four,  unrotated  factors.  The  authors 
found  a  high  correlation  (r  =  0.78)  between  the  overall  satisfaction  scores  in  the  long  and  short  versions. 

In  another  study,  Grogan,  Conner,  Willits,  and  Norman  (1995)  developed  a  patient  satisfaction 
questionnaire  to  measure  patients’  satisfaction  with  general  practitioners’  services.  They  collected  data  from 
1,  193  patients  identified  from  one  Norfolk  Practice  using  a  148-item  questionnaire.  Cronbach’s  alpha  for  the 
final  40-item  questionnaire  was  0.96.  Principal  component  analysis  of  the  40  items  revealed  five  factors  (with 
respective  Cronbach’s  alpha  score  in  parentheses):  doctors  (0.95),  access  (0.85),  nurses  (0.81)  appointments 
(0.83)  and  facilities  (0.73).  The  “doctors”  subscale  explained  34.9%  of  the  total  variance  and  had  the 
following  general  components:  information  giving  and  getting,  social  skills,  time  pressure  and  competence. 
There  was  a  statistically  significant  relationship  between  scores  on  each  of  the  subscales  and  the  general 
satisfaction  scores.  Women  and  older  individuals  tended  to  have  higher  satisfaction  ratings. 

Weiss  and  Senf  (1990)  developed  a  patient  satisfaction  survey  instrument  (adapting  some  items  from 
the  PSQ)  for  use  in  health  maintenance  organizations  that  would  help  predict  disenrollment  from  health  plans 
because  of  dissatisfaction.  The  questionnaire,  previously  pilot-tested,  included  satisfaction  questions  in  the 
following  areas:  physician  competence,  physician  personality,  doctor-patient  communication,  accessibility  and 
convenience  of  care,  facilities,  staff  attitudes  toward  patients,  and  availability  of  physicians,  hospital  and 
specialist  care.  Additional  questions  focused  on  personal  health,  frequency  of  health  services  utilization,  and 
sociodemographic  information.  Nine  items  were  found  to  predict  change  in  plans,  including:  “If  you  could 
change  health  plans  right  now,  what  would  you  do?”  “I  am  happy  with  the  coverage  provided  by  my  medical 
insurance  plan;”  ‘The  office  staff  treats  me  with  respect;”  “I’m  very  satisfied  with  the  medical  care  I  receive;” 
“The  office  hours  when  I  get  medical  care  are  good  for  me;”  “My  health  plan  is  primarily  concerned  about 
patients’  needs;”  “When  I  need  a  referral  to  a  specialist,  my  doctor  hesitates  to  send  me;”  “The  care  I  received 
from  the  doctors  in  the  past  year  is  just  about  perfect;”  “I  am  sometimes  ignored  by  the  staff  in  my  doctor’s 
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office;”  and  ‘There  are  things  about  the  medical  care  I  receive  that  could  be  better.”  (Cronbach’s  a  =  0.84; 
item  to  total  correlations  ranged  from  0.33  to  0.69). 

Steven,  Thomas,  Eckerman,  Browning,  and  Dickens  (1999)  developed  a  39-item  patient  satisfaction 
questionnaire.  Twenty  focus  groups  were  conducted  that  lead  to  the  identification  of  39  survey  items.  12,605 
respondents  from  133  Australian  general  practices  completed  this  survey.  The  authors  identified  three  major 
factors  related  to  overall  satisfaction:  interaction  between  patient  and  practitioner,  technical  “competence” 
(practitioner’s  ability  to  relate  to  different  segments  of  the  population)  and  accessibility  to  the  practice.  These 
three  factors  accounted  for  44.4%  of  variance  with  another  1 1 .5%  explained  by  minor  factors.  Characteristics 
associated  with  higher  satisfaction  were  older  age,  shorter  time  since  last  consultation,  longer  duration 
attending  the  practice,  regularly  attending  one  general  practice,  and  a  large  number  of  consultations  in  the  last 
month.  There  was  no  significant  difference  between  patients  attending  practices  of  different  sizes  in  their  level 
of  overall  satisfaction.  However,  patients  of  solo  practices  were  more  likely  to  be  satisfied  than  were  other 
patients. 

There  were  13  additional  studies  that  used  other  instruments  to  assess  patient  satisfaction  with  the 
provider(s)  or  type  of  provider.  Among  these  1 3  studies,  9  studies  presented  original  data  while  4  relied  on 
secondary  analyses  of  large  regional  or  national  studies  using  the  Consumer  Assessment  of  Health  Plans 
Survey,  the  RAND  Health  Insurance  Experiment,  HEDIS  2.0  data,  the  Health  Plan  Value  Check  Data,  the 
Dutch  National  Survey,  or  the  Commonwealth  Fund’s  Managed  Care  Survey  (Spoeri  et  al.,  1987;  Allen, 
1998;  Allen  1994;  Schauffler,  et  al.,  1996;  Weingarten,  Stone,  Green,  Pelter,  et  al.,  1995).  Below  are  the 
survey  instruments  and  the  studies  in  which  they  are  cited  (more  specific  information  can  be  found  in 
Appendix  B  and  in  the  literature  review  article  summaries). 


•  Trust  in  Physicians  Scale  (Thom,  Ribisl,  Stewart,  Luke,  Stanford  Trust  Study  Physicians,  1999) 

•  Medical  Preference  Survey  (Linn,  1984) 

•  Primary  Care  Assessment  Survey  (Taira,  Safran,  Seto,  Roger,  et  al.,  1997;  Safran,  et  al.,  1998) 

•  Dialog  Patient  Satisfaction  Survey  (Poulton,  1996) 

•  Measures  of  System  Performance  (Nelson  1995) 

•  Davis  Observation  Code  (Robbins,  et  al.,  1993) 

Unpublished  Studies  Assessing  Quality  of  Care  in  Physician  Groups 

Two  unpublished  studies  presented  data  collected  as  part  of  The  Physician  Value  Check  Survey  (PVCS). 
The  PVCS  is  a  longitudinal  study  coordinated  by  the  Pacific  Business  Group  on  Health  (PBGH)  (Pacific 
Business  Group  on  Health,  1996  &  1999)  which  is  being  conducted  with  managed  care  patients  of 
participating  sites  as  well  as  non-managed  care  patients  of  preferred  provider  organizations  (PPOs)  and 
includes  data  from  two  points  in  time-  1996  and  1998.  The  PVCS  is  a  groundbreaking  study  in  several 
respects.  First,  it  is  the  only  documented  study  that  measures  and  reports  on  performance  of  physician  groups 
rather  than  at  the  health  plan  level.  Second,  the  survey  provides  physician  groups  involved  in  managed  care 
with  comparative  data  to  benchmark  their  individual  performance  across  a  number  of  different  dimensions  of 
care.  Finally,  the  survey  provides  California  consumers  access  to  satisfaction  and  quality  data  about  their 
physician  groups. 

PBGH  conducted  an  initial  survey  in  1996  in  collaboration  with  The  Medical  Quality  Commission 
(TMQC)  and  a  follow-up  survey  in  1998.  The  1996  PVCS  surveyed  60,000  patients  from  55  managed  care 
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physician  groups  in  California,  Oregon  and  Washington.  The  1998  PVCS  includes  data  on  30,774  patients 
from  57  managed  care  physician  groups  (some  who  were  previous  participants)  and  2  PPO  provider 
organizations.  At  both  times,  patients  were  administered  the  PVCS  and  physicians  at  those  participating  sites 
were  asked  to  complete  a  Physician  Group  Attributes  Survey  which  described  selected  characteristics  of  their 
organization. 

The  PVCS  builds  on  prior  survey  efforts  by  both  the  TMQC  and  PBGH.  Questions  for  the  PVCS  were 
drawn  from  existing  survey  instruments  (e.g.,  MOS,  the  American  Association  of  Health  Plans  (AAHP),  and 
the  RAND  Health  Insurance  Experiment  (HIE))  and  adapted  to  measure  patient  satisfaction  at  the  medical 
group  and  IPA  level.  The  PVCS  assesses  four  main  areas:  patient  satisfaction  with  care  provided  by  the 
patient’s  doctors;  longitudinal  change  in  self-reported  health  and  functional  status;  self-reported  receipt  of 
selected  preventive  services;  and  self-reported  processes  of  care  received  for  two  chronic  conditions- 
hypertension  and  high  cholesterol.  The  1998  PVCS  contained  nearly  all  the  questions  from  the  1996  survey 
but  included  a  broader  set  of  questions  on  access  to  and  satisfaction  with  specialty  care,  a  measure  of 
disability  days,  questions  to  assess  the  prevalence  of  depression  and  care  for  mental  health  problems,  several 
items  to  assess  receipt  of  colorectal  cancer  screening,  and  process  of  care  measures  for  medications. 

In  this  study  summary  satisfaction  scales  were  created  by  averaging  together  items  that  measured  the  same 
concept  using  initial  specifications  from  the  New  England  Medical  Center  and  exploratory  factor  analysis  and 
substantive  considerations.  The  following  satisfaction  scales  were  developed:  access  to  care  (7  items, 
alpha=0.89,  RTT=0.93),  promptness  of  care  (2  items,  alpha=0.75,  RTT=0.94),  cost  of  care  (2  items, 
alpha=0.82,  RTT=0.94),  technical  quality  (3  items,  alpha=0.85,  RTT=0.94),  interpersonal/  communication  (6 
items,  alpha=0.95,  RTT=0.94),  courtesy  of  office  staff  (1  item,  RTT=0.90),  health  plan  services  (1  item, 
RTT=0.93),  overall  satisfaction  with  doctor  (1  item,  RTT=0.88),  overall  rating  of  care  (2  items,  alpha-0.92), 
and  overall  satisfaction  with  health  plan  (1  item,  RTT=0.93). 

For  casemix  adjustment,  the  satisfaction  scales  were  regressed  on  age,  gender,  education,  income  and 
selected  chronic  conditions.  In  the  1996  study  satisfaction  was  not  regressed  on  race/ethnicity  since  the 
assumption  was  that  people  of  different  race/ethnicity  groups  are  entitled  to  the  same  quality  of  health  care.  In 
the  1998  survey,  race/ethnicity  was  included  in  the  regression  model.  The  satisfaction  scales  were  also 
regressed  on  length  of  time  receiving  care  from  the  physician  group,  utilization  and  health  status. 

The  1996  study  found  differences  in  self-reported  satisfaction  on  several  areas  based  on  geographic 
location  of  the  group  practices  (e.g.,  Southern  versus  Northern  California  and  Northwest  region)  and  the  type 
of  practice  setting  (managed  care  versus  PPO).  In  the  1996  PVCS,  IPAs  received  higher  scores  on  satisfaction 
than  did  medical  groups.  With  respect  to  access,  no  significant  differences  were  found  in  the  1996  survey  and 
few  significant  differences  were  found  in  the  1998  PVCS  by  regions  and  by  types  of  organizations. 


Domains  of  Care  Assessed  in  all  the  Studies  Reviewed 

Dimensions  included  in  the  articles  were  identified  through  an  analysis  of  the  survey  items  assessing 
consumer  assessments  of  health  care  providers.  In  some  of  the  studies,  we  were  able  to  review  the  actual 
questionnaire  items,  while  in  other  studies  we  had  to  rely  on  published  information.  Table  1  lists  dimensions 
of  patient  satisfaction  measured  in  studies  reviewed  for  the  PG-CAS  instrument  development  effort.  These 
domains  are  briefly  summarized  below. 
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Domains  Related  to  Physician  Practice  or  Health  Plan 


•  Finances:  (ability  to  pay  for  services  or  arrange  for  payment) 

•  Availability  of  resources:  (availability  of  hospitals,  clinics,  staff,  specialists,  resources  that  patients  can  use) 

•  Efficacy  of  care:  (measured  in  terms  of  perceptions  regarding  the  usefulness  or  helpfulness  of  medical  care 
providers  and  specific  treatment  regimens  in  improving  or  maintaining  health  status) 

•  Accessibility/  access  to  car e/  convenience:  factors  involved  in  getting  or  receiving  medical  care.  Such 
items  can  include  ease  of  getting  an  appointment  and  checking  out  after  an  office  visit;  convenience  of 
appointments,  office  hours,  office  location,  and  parking;  time  in  waiting  rooms  and  exam  rooms; 
punctuality  of  doctors;  and  ease  of  obtaining  medical  advice  after  hours.  The  majority  of  the  studies 
included  questions  related  to  access  to,  accessibility,  and  convenience  of  a  health  care  facility  or  health 
plan. 

•  Continuity:  (the  extent  to  which  the  patient  sees  his/her  personal  physician  most  of  the  time  and  knowing 
the  doctor  and  staff;  the  regularity  of  care  from  the  same  facility,  provider,  location).  Some  of  the  studies 
studied  areas  related  to  continuity  of  care,  most  specifically  in  the  area  of  provision  of  primary  care  within 
a  healthcare  setting. 

•  Physical  environment:  satisfaction  with  the  pleasantness  of  the  office  or  the  physical  environment  in  which 
the  care  is  delivered  (e.g.,  pleasantness  of  atmosphere,  attractiveness  of  waiting  rooms,  orderly  facilities 
and  equipment,  etc) 

Domains  Related  to  Physician  and/or  Other  Health  Care  Provider 

•  Professional  competence/  quality:  (competence  of  providers  and  their  adherence  to  high  standards  of 
diagnosis  and  treatment).  Several  of  the  studies  we  reviewed  assessed  the  provider’s  competence  and 
quality  in  the  provision  of  medical  care.  Some  studies  had  specific  questions  focusing  on  the  doctor’s 
knowledge,  provision  of  advice  by  physician  regarding  a  medical  problem,  thoroughness  of  encounter, 
referrals  to  specialists  for  complex  problems,  provision  on  prescriptions 

•  Competence  in  emotional  domain:  It  involves  patient’s  perception  of  the  provider’s  understanding  of  the 
importance  of  their  emotional  status.  A  few  studies  had  questions  related  to  the  doctor’s  empathy, 
concern,  worry,  and  respect  for  patient’s  concerns. 

•  Interpersonal  manner:  It  involves  rating  the  interpersonal  skills  of  physicians,  nurses,  receptionists,  billing 
personnel,  and  other  office  staff.  Several  studies  assessed  this  domain  and  had  questions  related  to  the 
doctor’s  or  staffs  respect,  concern,  communication  with  patient  and  how  well  they  related  to  patient. 

•  Humaneness  of  the  physician  /  art  of  care:  (factors  assessing  the  caring  shown  towards  patients).  A  few 
studies  had  questions  related  to  the  doctor’s  empathy,  concern,  worry,  and  respect  for  patient’s  concerns 

•  Efficacv/outcomes:  assesses  the  extent  to  which  a  patient  feels  better  after  seeing  the  physician  and  the 
degree  of  comfort  with  the  recommended  treatment. 
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•  Mutual  understanding:  (patient’s  perception  of  whether  they  are  treated  as  equal  partners  when  decisions 
are  made  regarding  their  health  care). 

Uses  of  Consumer  Assessments  of  Health  Care 

Patient  assessments  of  health  care  are  important  not  only  as  a  measure  of  the  quality  of  care  patients 
receive,  but  also  in  identifying  potential  areas  for  improving  the  content  of  care  provided  by  physicians. 
Research  has  suggested  that  improving  patient  satisfaction  with  physicians  increases  the  likelihood  that  a 
patient  will  return  to  a  given  health  care  provider,  which  is  especially  important  for  health  care  providers  in  an 
increasingly  competitive  medical  care  marketplace.  Many  of  the  articles  that  we  reviewed  looked  at  patient 
satisfaction  as  a  dependent  factor,  contingent  upon  “quality”  of  care  in  other  aspects.  Several  studies  even 
focused  on  primary  care  factors  and  their  relation  to  satisfaction. 

Satisfaction  data  have  been  used  most  frequently  to  evaluate  health  and  medical  care  services.  For 
example,  Isenberg  and  Stewart  (1998)  used  patient  satisfaction  data  (using  the  adapted  MOS  VSQ)  to  assess 
quality  improvement  at  community-based  medical  practices  where  they  had  implemented  a  quality 
improvement  protocol  in  random  sites.  Explanation  given  by  the  doctor,  time  with  the  doctor,  and  personality 
were  the  three  physician  attributes  improved  by  the  intervention.  In  another  study  (Bush,  Cherkin,  &  Barlow, 
1993),  the  authors  assessed  whether  patient  satisfaction  was  related  to  physicians’  confidence  in  their  abilities 
to  effectively  manage  low  back  pain  (LPB)  and  to  examine  their  attitudes  about  patients  with  LBP.  The 
confidence  and  attitudes  of  PCPs  were  determined  using  self-administered  questionnaires.  Patient  satisfaction 
was  measured  with  a  previously  validated  instrument  that  was  administered  3  weeks  after  their  clinic  visit.  The 
instrument  measures  three  dimensions  of  patient  satisfaction:  information  received  from  providers  (3  items); 
empathy  or  caring  (4  items);  and  the  perceived  effectiveness  of  therapy  (3  items).  Data  on  21  physicians  and 
270  patients  were  included.  Provider  attitude  was  not  related  to  any  patient  satisfaction  scales,  but  provider 
confidence  was  significantly  related  to  patient  satisfaction  with  information,  but  not  to  patient  satisfaction  with 
the  caring  nature  of  the  provider. 

In  addition,  a  few  studies  examined  the  relationship  between  preventive  care,  health  education  or 
health  promotion  and  patient  satisfaction  with  their  physician  suggesting  that  there  is  a  positive  association. 
For  example,  Schauffler,  Rodriguez  and  Milstein  ( 1 996)  found  that  patients  who  self-reported  that  a  physician 
and/or  other  health  care  provider  discussed  health  education  topics  with  them  within  the  past  3  years,  were 
twice  as  likely  to  be  satisfied  with  the  provider  compared  to  those  that  had  not  received  patient  education 
regardless  of  the  health  plan  in  which  they  were  enrolled.  In  this  study,  age  and  education  were  not 
statistically  related  to  patient  satisfaction,  although  sex,  health  status,  employee  satisfaction  with  out-of- 
pocket  costs  were  positively  associated  with  patient  satisfaction.  Korke,  Solberg,  Brekke  and  colleagues 
(1997)  found  that  self-reports  of  being  advised  to  have  a  preventive  service  when  due  correlated  with  higher 
levels  of  satisfaction  only  and  with  general  satisfaction  index.  Finally,  in  another  study  (Weingarten,  et  al., 
1995)  the  authors  found  a  significant  relationship  between  patient  satisfaction  and  provision  of  some  but  not 
all  preventive  care  services. 

Other  studies  have  assessed  the  impact  of  various  clinic  characteristics  and  some  provider 
characteristics  that  affect  satisfaction.  In  an  early  study,  Hulka  and  colleagues  (1975)  found  that  although 
attitudes  of  consumers  were  more  favorable  with  respect  to  professional  competency  and  personal  qualities  of 
providers,  accessibility,  including  costs  and  convenience,  were  less  highly  regarded.  In  this  same  study,  the 
author  stated  that  six  factors  accounted  for  60%  of  variance  in  satisfaction  and  included:  physician  conduct 
(41.4%),  continuity  of  care,  accessibility,  availability  of  hospitals/  specialists,  completeness  of  facilities,  and 
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availability  of  family  doctors.  In  a  more  recent  study,  Schauffler  and  colleagues  (1996)  found  that  a  patient- 
centered  practice  style  and  enrollment  in  a  PPO  were  positively  associated  with  patient  satisfaction.  These 
authors  also  found  that  communication,  affective  care,  and  technical  competence  correlated  highly  with  each 
other.  Saffan  and  colleagues  (1998)  found  that  access,  continuity,  comprehensiveness,  integration,  clinical 
interaction,  interpersonal  treatment  and  trust  were  predictors  most  strongly  related  to  patient  satisfaction. 
Handler  and  colleagues  (1998),  for  example,  found  that  among  African  American  and  Mexican  American 
women  seeking  prenatal  care,  the  following  characteristics  to  be  associated  with  increased  satisfaction:  having 
procedures  explained  by  the  provider,  short  waiting  times  at  site,  availability  of  ancillary  services,  and  having  a 
male  prenatal  care  practitioner.  Chung  (1999)  found  that  patient  satisfaction  was  most  strongly  related  to 
efficient  clinic  operation  and  the  quality  of  the  patient-physician  interaction.  Finally,  other  researchers  (Thom 
et  al.,  1999)  have  found  that  how  the  physician  was  chosen,  patient  preference  for  autonomy,  self-care,  and 
expectations  regarding  proportion  of  care  provider  should  provide  were  significantly  related  to  satisfaction. 

Satisfaction  as  a  Predictive  Factor 


Several  studies  have  used  satisfaction  measures  to  predict  disenrollment  and  adherence  to  treatment. 
One  study  examined  the  ability  of  a  satisfaction  measure  to  predict  who  was  most  likely  to  disenroll  from  a 
particular  health  plan  (Weiss  &  Senf,  1990).  In  this  study  the  authors  developed  a  survey  instrument  with 
specific  applicability  to  prepaid  health  care  systems  that  included  critical  variables  that  would  predict 
disenrollment  from  health  plans  because  of  dissatisfaction.  Analysis  was  done  on  2,365  members  who  had 
returned  both  pre-  and  post-enrollment  questionnaires  and  who  stated  they  were  enrolled  in  one  of  three 
HMO  options  offered  by  the  university.  Through  discriminant  function  analysis,  nine  items  on  the 
questionnaire  were  found  to  predict  change  in  plans  (Cronbach’s  a  =  0.84;  item  to  total  correlations  ranged 
from  0.33  to  0.69).  Another  study  assessed  patient  compliance  /  adherence  with  treatment  at  6  months  follow¬ 
up  (Thom,  et  al.,  1999)  by  using  the  Trust  in  Physician  Scale  and  interviewing  343  male  and  female  patients 
seen  in  20  community-based  practices  of  primary  care  physicians  in  Northern  California.  Patients  completed 
the  1 1-item  Trust  in  Physicians  Scale  at  enrollment  as  well  as  questions  on  demographics,  preferences  for 
care,  desire  for  autonomy  during  visit,  length  of  relationship  with  their  current  physician,  how  current  doctor 
was  chosen,  and  expected  proportion  of  care  to  be  received  from  current  doctor.  A  post-  visit  questionnaire 
assessed  the  humanistic  physician  behaviors  during  the  visit  and  satisfaction  with  care  received  from  the 
physician  (assessed  using  a  subset  of  13  questions  from  the  Consumer  Satisfaction  Survey  which  assessed 
technical  quality,  communication,  interpersonal  care,  and  outcomes).  Patients  completed  the  Trust  in 
Physician  Scale  by  mail  survey  1  month  after  their  enrollment  visit.  At  6  months,  patients  were  surveyed  by 
mail  to  ascertain  continuity  with  their  physician,  adherence  to  prescribed  medications  and  their  satisfaction 
with  the  care  from  their  physician,  and  general  measure  of  interpersonal  trust  between  patient  and  provider. 
The  Trust  in  Physician  scale  demonstrated  high  predictive  validity  with  all  outcome  measures  (continuity,  self- 
reported  adherence  with  medications,  and  satisfaction)  at  6  months  even  after  adjusting  for  various  co variates 
and  even  after  controlling  for  baseline  satisfaction. 

Casemix  Adjustment 

Sociodemographic  correlates  of  patient  satisfaction  identified  in  this  literature  review  include  the  following: 

Age:  In  several  studies  we  found  a  positive  relationship  between  age  and  general  satisfaction  (e.g., 

Weingarten,  et  al,  1995;  Shmittdiel  et  al.,  1997;  Grogan  et  al.,  1995);  with  the  physician’s  skills  and 

competence  (e.g.,  Singh  et  al.,  1999);  and  with  the  physician’s  courtesy  and  consideration  (e.g.,  Singh 
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et  al.,  1999).  In  other  studies,  however,  age  was  not  statistically  related  to  satisfaction  (Schauffler, 
1996). 

Education:  Many  articles  have  noted  that  less  educated  individuals  tend  to  be  less  satisfied  than 
higher  educated  individuals  (e.g.,  Singh  et  al.,  1999).  Other  authors  (e.g.,  Chung,  1999;  Schauffler, 
1996),  for  example,  did  not  find  any  significant  relationship  between  education  and  satisfaction. 
However,  other  authors  (e.g.,  Safran  et  al.,  1998)  found  a  positive  and  significant  relationship  between 
education  and  satisfaction. 

Income/ SES:  DiMatteo  &  Hays  (1980)  did  not  find  any  significant  correlations  between  SES  of 
patients  and  their  general  satisfaction  with  care  or  their  perceptions  of  provider  conduct.  Schmittdiel 
and  colleagues  (1997)  found  that  decreased  SES  was  associated  with  increased  patient  satisfaction  in 
all  their  patients. 

Marital  status:  One  study  found  a  positive  association  between  marital  status  (being  married)  and 
higher  satisfaction  (Tessler,  and  Mechanic,  1975).  This  was  the  only  study  that  correlated  this 
sociodemographic  factor  with  satisfaction. 

Occupational  level:  Some  authors  found  that  individuals  with  higher  skills  tended  to  be  more  satisfied 
in  general  or  with  specific  aspects  of  care  (e.g.,  technical  competence  of  provider). 

Race/ethnicity:  Results  regarding  relationships  between  satisfaction  and  race/ethnicity  varied  among 
the  studies.  Hulka  and  his  collaborators  (1975)  for  example  found  that  young  Black  adults  expressed 
more  dissatisfaction  with  their  physician’s  personal  qualities  than  other  groups.  Schmittdiel  and 
colleagues  (1997)  also  found  that  Blacks  were  less  satisfied  with  medical  care  compared  to  Whites.  In 
another  study,  the  authors  found  that  Asian  American  ethnicity  was  significantly  associated  with  lower 
satisfaction  scores  (Taira  et  al.,  1997).  Morales  and  colleagues  (1999,  2000)  found  that  Hispanics 
were  significantly  more  dissatisfied  with  care  than  whites.  Morales,  Reise,  and  Hays  (2000)  used  data 
from  the  Unified  Medical  Group  Association  and  found  that  Hispanics  were  significantly  more 
dissatisfied  with  care  then  whites  (effect  size=0.27;  P  <  0.05)  on  7  of  the  9  items  dealing  with 
interpersonal  and  technical  aspects  of  medical  care.  From  this  study  they  concluded  that  valid 
comparisons  between  whites  and  Hispanics  are  possible  and  that  disparities  in  satisfaction  ratings 
between  whites  and  Hispanics  should  not  be  ascribed  to  measurement  bias  but  should  be  viewed  as 
arising  from  actual  differences  in  experiences  with  care.  However,  some  studies  (e.g.,  Chung  1999) 
did  not  find  any  significant  relationship  between  ethnicity  and  satisfaction. 

Sex  /  gender:  There  were  mixed  results  regarding  relationship  of  sex  and  satisfaction.  Although  many 
studies  did  find  higher  satisfaction  among  women  (Hulka  et  al.,  1975;  Grogan  et  al.,  1995;  Schauffler, 

1996) ,  other  studies  did  not  find  any  significant  relationship  (Chung,  1999;  Weingarten,  et  al.,  1995) 
and  even  some  found  that  men  had  higher  satisfaction  scores  than  women  (e.g.,  Schmittdiel  et  al., 

1997) . 
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Description  of  databases  used: 
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MEDLINE  (MEDlars  onLINE)  is  the  National  Library  of  Medicine's  (NLM)  premier  bibliographic  database  covering  the  fields 
of  medicine,  nursing,  dentistry,  veterinary  medicine,  the  health  care  system,  and  the  pre-clinical  sciences.  It  contains 
bibliographic  citations  (e.g.,  authors,  title,  and  journal  reference)  and  author  abstracts  from  over  3,900  biomedical  journals 
published  in  the  United  States  and  70  foreign  countries  during  the  current  four  years.  It  also  contains  over  9  million  records 
dating  back  to  1966  and  has  worldwide  coverage,  but  88%  of  the  citations  in  the  current  MEDLINE  are  from  English-language 
sources  and  76%  have  English  abstracts.  Citations  for  MEDLINE  are  created  by  the  National  Library  of  Medicine,  International 
MEDLARS  ‘partners,  and  cooperating  professional  organizations. 


HEALTHSTAR  contains  relevant  bibliographic  records  from  MEDLINE  (1975  to  present)  and  unique  records  from  three 
sources:  (1)  records  emphasizing  health  care  administration  selected  and  indexed  by  the  American  Hospital  Association  (AHA); 
(2)  records  emphasizing  health  planning  from  the  National  Health  Planning  Information  Center  (only  in  the  backfile);  and  (3) 
records  emphasizing  health  services  research,  clinical  practice  guidelines,  and  health  care  technology  assessment  selected  and 
indexed  through  NLM's  National  Information  Center  on  Health  Services  Research  and  Health  Care  Technology  (NICHSR).  It 
includes  journal  articles,  technical  and  government  reports,  meeting  papers  and  abstracts,  books  and  book  chapters. 


ERIC  is  the  Educational  Resources  Information  Center  (ERIC),  a  federally-funded  national  information  system  that  provides  a 
variety  of  services  and  products  on  education-related  issues.  The  ERIC  database,  the  world's  largest  source  of  education 
information,  contains  more  than  950,000  abstracts  of  documents  and  journal  articles  on  education  research  and  practice. 
Abstracts  in  the  ERIC  Database  are  available  in  printed  version  in  Resources  in  Education  and  Current  Index  To  Journals  in 
Education .  The  database  is  updated  monthly,  ensuring  that  the  information  is  timely  and  accurate. 


JSTOR  is  a  database  funded  by  the  Mellon  Foundation  and  contains  full  text  electronic  versions  of  articles  from  1 17  journals  in 
the  following  fields:  African-American  Studies,  Anthropology,  Asian  Studies,  Economics,  Education,  Finance,  Literature, 
Mathematics,  Philosophy,  Science,  Population  Studies,  Sociology  and  Statistics.  Not  all  of  these  disciplines  are  relevant  for  our 
study,  but  some  may  have  useful  citations  that  are  not  found  in  the  medical  literature.  Since  it  is  a  new  service,  only  articles  from 
the  last  3-5  years  are  available. 


UNCOVER  is  a  database  of  current  article  information  taken  from  over  1 8,000  multidisciplinary  journals.  UnCover  contains 
brief  descriptive  information  for  over  8,800,000  articles  which  have  appeared  since  fall  1988.  UnCover  is  easy  to  use,  with 
keyword  access  to  article  titles  and  summaries. 
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Access  to  the  Databases: 


The  table  below  summarizes  the  WWW  search  sites  that  we  used  to  access  the  databases: 


Database 

Site  Name 

WWW  Address: 

Years 

Advantages  /  Disadvantages 

MEDLINE 

Grateful  Med 

http://igm.nlm.nih.gov 

66-99 

Official  gov’t  search  engine  with  many 
help  and  support  files.  Easy  to  use 
interface. 

MEDLINE 

Infotrieve 

http://www.infotrieve.eom/f 

reemedline/ 

66-99 

Easy  to  use  interface  and  easily  read 
citations 

MEDLINE 

HealthGate 

http://www.healthgate.com/ 

medline/adv-medline.shtml 

66-99 

Easy  to  use  interface.  Has  special 
advanced  search  functions. 

HEALTH 

STAR 

Grateful  Med 

http://igm.nlm.nih.gov 

75-99 

Official  gov’t  search  engine  with  many 
help  and  support  files.  Easy  to  use 
interface. 

HEALTH 

STAR 

HealthGate 

http://www.healthgate.com/ 

healthstar/adv- 

healthstar.shtml 

75-99 

Easy  to  use  interface.  Has  special 
advanced  search  functions. 

ERIC 

AskERIC 

http://ericir.syr.edu/Eric/ 

66-99 

AskERIC  is  the  official  ERIC  search 
engine. 

JSTOR 

JSTOR 

http://www.jstor.org/jstor/ 

94-99 

Provides  full  text  articles  for  those  in 
database.  Many  social  science  journals. 

UNCOVER 

UnCover  Web  2.0 

http://uncweb.carl.org/ 

88-99 

Basic  search  features  offer  a  broad 
overview  of  titles  in  many  different  fields. 

Note:  The  short-list  of  MEDLINE  and  HEALTHSTAR  search  engines  was  chosen  by  reviewing  many  different  WWW  sites  and 
selecting  those  that  had  the  most  hyperlinks  pointed  to  them  and  a  diversity  of  features.  The  search  engines  for  ERIC,  JSTOR, 
and  UNCOVER  are  the  only  ones  that  are  publicly  available. 
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Consumer  Assessment  Survey 
Literature  Review:  Keywords  and  Combinations 


Goals: 

Search  for  literature  and  write  a  review  on: 

♦  Consumer  assessment  of  providers  at  the  medical  group  level 


Keywords:  Consumer  assessment  of_ providers 


Physician 

Physician  Group 

Medical  Group 

IPA 

Provider 

Consumers 

Patient 

Survey 

Instrument 

Questionnaire 

Assessment 

Evaluation 

Satisfaction 

Patient  Satisfaction 

Access  to  Specialists 

Process  of  Care 

Health  Outcomes 

Combinations  of  keywords  used  for  first  stage  literature  review: 

Consumer  assessment  of  providers 

[Physician  OR  Physician  Group  OR  Medical  Group  OR  IP  A]  AND 
[Physician  OR  Physician  Group  OR  Medical  Group  OR  IP  A]  AND 
[Physician  OR  Physician  Group  OR  Medical  Group  OR  IP  A]  AND 
[Physician  OR  Physician  Group  OR  Medical  Group  OR  IP  A]  AND 
[Physician  OR  Physician  Group  OR  Medical  Group  OR  IP  A]  AND 

[Physician  OR  Physician  Group  OR  Medical  Group  OR  IPA]  AND 
[Physician  OR  Physician  Group  OR  Medical  Group  OR  IPA]  AND 
[Physician  OR  Physician  Group  OR  Medical  Group  OR  IPA]  AND 
[Physician  OR  Physician  Group  OR  Medical  Group  OR  IPA]  AND 
[Physician  OR  Physician  Group  OR  Medical  Group  OR  IPA]  AND 

[Physician  OR  Physician  Group  OR  Medical  Group  OR  IPA]  AND 
[Physician  OR  Physician  Group  OR  Medical  Group  OR  IPA]  AND 
[Physician  OR  Physician  Group  OR  Medical  Group  OR  IPA]  AND 
[Physician  OR  Physician  Group  OR  Medical  Group  OR  IPA]  AND 
[Physician  OR  Physician  Group  OR  Medical  Group  OR  IPA]  AND 

[Physician  OR  Physician  Group  OR  Medical  Group  OR  IPA]  AND 
[Physician  OR  Physician  Group  OR  Medical  Group  OR  IPA]  AND 
[Physician  OR  Physician  Group  OR  Medical  Group  OR  IPA]  AND 
[Physician  OR  Physician  Group  OR  Medical  Group  OR  IPA]  AND 
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